Augmented reality music composition

ABSTRACT

Systems and methods for re-arranging a musical composition using augmented reality are disclosed. A user can be provided with one or more physical image markers representative of at least one of a musical entity and a musical style in which a musical track has been pre-recorded. Upon capturing a visual scene in which one or more of these physical image markers are included, an augmented reality version of the visual scene is displayed along with audio corresponding to the musical style(s) and/or entities represented by the one or more physical image markers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/089,816, filed on Apr. 4, 2016, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to augmented and/or virtualreality presentations.

DESCRIPTION OF THE RELATED ART

Many user devices include some form of camera device which may be usedto capture an image or a series of images of a physical, real-worldscene. The captured images can then be displayed, for instance, on adisplay of the user device. Certain user devices, by way of an augmentedreality application or functionality, may be configured to insertvirtual objects into the captured images before and/or while the imagesare displayed.

BRIEF SUMMARY OF THE DISCLOSURE

In accordance with one embodiment, a computer-implemented method,comprises capturing a visual scene, and identifying a physical imagemarker within the visual scene. The method further comprises augmentingthe visual scene with the augmented reality version of the physicalimage marker. Further still, the method comprises outputting anaugmented reality version of the visual scene for display in conjunctionwith audio output based upon at least one musical characteristic of theaugmented reality version of the physical image marker.

In accordance with another embodiment, an apparatus, comprises at leastone camera capturing a visual scene, and an augmented reality componentidentifying a physical image marker within the visual scene, augmentingthe visual scene with the augmented reality version of the physicalimage marker. The apparatus further comprises a display displaying anaugmented reality version of the visual scene, and one or more speakersoutputting audio in conjunction with the augmented reality version ofthe visual scene and based upon at least one musical characteristic ofthe augmented reality version of the physical image marker.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The figures are provided for purposes of illustration only andmerely depict typical or example embodiments.

FIG. 1 is an operational flow diagram illustrating an example processfor augmented reality music composition in accordance with variousembodiments.

FIG. 2A illustrates an example augmented reality device in accordancewith various embodiments.

FIGS. 2B-2D illustrate example augmented reality interactions andpresentations in accordance with various embodiments.

FIG. 3 is an example computing module that may be used to implementvarious features of embodiments described in the present disclosure.

The figures are not exhaustive and do not limit the present disclosureto the precise form disclosed.

DETAILED DESCRIPTION

Existing technology for music composition typically focuses on softwaresolutions for professionals. For example, sound mixing software, MIDI,and other tools allow a user to mix audio tracks, activate anddeactivate instruments, modulate sound, etc. However, these softwaresolutions often rely on traditional computer interfaces such as acomputer display through which user interface elements like buttons andsliders, files, menus, etc. are presented. The majority of musiccomposition software solutions targets professionals. The few aimed athobbyists still rely on complex user interfaces having a steep learningcurve. Leveraging augmented reality technologies in the context of musiccomposition would improve the user experience associated withconventional music composition software, and would also provide a rich,creative, and imaginative experience for experienced and inexperiencedusers alike, such as children. Such a user experience allows for activediscovery which can provide a rich learning experience and provide afoundation for creative problem solving skills. Physical interactionwith one's surroundings is a critical component of the experientiallearning that lies at the heart of creative play, and provides analternative to passively consuming content.

Accordingly, various embodiments leverage an augmented realityexperience to provide a bridge between real-world musical activities anddigital enhancements. In particular, various embodiments provide anaugmented reality music composition experience that gives users thefreedom to explore different music styles/genres and musical instrumentsto create unique versions of musical compositions.

For example, a user may be provided with one or more physical imagemarkers, each representing at least a musical entity, such as avocalist, instrument, or apparatus. In some embodiments, the musicalentity may have an associated music style or genre. Examples of musicalentities represented by such physical image markers, may include, butare not limited to percussion instruments such as drums, brassinstruments such as trumpets, stringed instruments such as violins,guitars, a singer or movie character that sings, and a DJ turntable. Thephysical image marker itself may be a physical card, chip, etc. In someembodiments, physical image markers may be collectable trading cardsand/or stickers users may purchase, exchange, trade, etc. In someembodiments, physical image markers can be used to represent musicians,such as vocalists from popular bands and the like. In some embodiments,the physical image markers may be representative of multiple musicalentities, such as a vocalist that plays a guitar, for example.

The user may view or capture a real-world scene via an augmented realitydevice. Upon placing a physical image marker on a physical boardassociated with an augmented reality application (within the real-worldscene) or simply upon recognition of the physical image marker (withinthe real-world scene), an augmented version of the instrument, entity,or apparatus represented by the physical image marker is shown andcorresponding audio may be played on the augmented reality device.

FIGS. 1, 2A, and 2B will be described in conjunction with each other,where FIG. 1 illustrates example processes performed in accordance withvarious embodiments for effectuating an augmented reality musiccomposition experience. FIG. 2A is a block diagram of an exampleuser/augmented reality device that may be used to achieve the augmentedreality music composition experience, and FIG. 2B illustrates an exampleof an augmented reality music composition experience in accordance withone embodiment.

At operation 100 of FIG. 1, a visual scene is captured. The visual scene(an example of which is illustrated in FIG. 2B) may be captured by someform of augmented reality device. FIG. 2A is an example representationof such an augmented reality device 200, which may be implemented assolely an augmented reality device or as part of a user device, such asa smart phone or tablet PC. As shown, the augmented reality device 200includes an augmented reality component 202, one or more cameras 204, adisplay device 206, one or more speakers 208 and an optionalaccelerometer 210.

As used herein, an augmented reality device refers to any device capableof displaying a real-time view of a physical, real-world environmentwhile altering elements within the displayed view of the environment. Assuch, unlike a virtual reality device which displays a view of virtualworld, an augmented reality device displays a view of the real world butaugments (e.g., adds or modifies) elements using computer graphicstechnology. Such an augmented reality device may include and/or becommunicatively coupled to a camera device (or multiple camera devices)used to capture a view of the real-world environment and may furtherinclude computer software and/or hardware configured to augment elementsof the captured scene. For example, an augmented reality device couldcapture a series of images of a coffee cup sitting on top of a table,modify the series of images so that the coffee cup appears as ananimated cartoon character and display the modified series of images inreal-time to a user. As such, when a user looks at the augmented realitydevice, the user sees an augmented view of the physical real-worldenvironment in which the user is located.

The one or more cameras 204 may include cameras for capturing the visualscene. As used herein, a visual scene refers to a view(s) of thereal-world environment in which the augmented reality device 200 isbeing used. For instance, a visual scene may be a series of images of areal-world environment. In the example illustrated in FIG. 2B, thevisual scene captured by augmented reality device 200 includes aphysical image marker 222. Generally, the accelerometer 210 is a devicecapable of measuring the physical acceleration of the augmented realitydevice 200. The augmented reality component 202 may use theaccelerometer 210 to determine when the position of the augmentedreality device 200 is changing, for example, which could indicate thatthe position of the augmented reality device 200 relative to one or morephysical image markers, such as physical image marker 222, is changing.Augmented reality component 202 may also on its own or with assistancefrom accelerometer 210, determine the positioning of a physical imagemarker relative to augmented reality device 200.

Returning to FIG. 1, at operation 102, the augmented reality component202 is configured to identify or otherwise recognize physical imagemarkers within a visual scene which, as described above, can be a seriesof frames captured using one or more cameras 204 and to transform thedepiction of the visual scene on the augmented reality device 200 basedon one or more characteristics of the physical image markers recognizedwithin the visual scene. That is, at operation 104, the visual scene isaugmented with an augmented reality version of the physical imagemarker.

In addition to augmentation from a visual perspective, as alluded toabove, a musical aspect is also introduced by the augmented realityversion of the physical image marker. Accordingly, at operation 106, thevisual scene is output for display in conjunction with audio output (viaone or more speakers 208) based upon at least one musical characteristicof the augmented reality version of the physical image marker.

In particular, when composing music, a composer develops melodies andrhythms, decides on keys, scales, chords, and tempi, and choosesinstruments for each part of a musical composition. Most often, thecomposer will adhere to a specific music style while writing the musicalcomposition. While many styles or genres of music exist, commoncategories of styles may include Alternative, Blues, Classical, Dance,Hip-Hop, Pop, Jazz, Soul, Rock, and World. Such music styles oftendictate specific rhythmic elements, instruments, and scales.Nevertheless, most any musical composition can be re-arranged withdifferent instruments, tempi, and rhythm to express a different musicalstyle. For example, playing a known Punk-Rock song arranged for electricguitars and standard drums can be re-arranged in a Reggae style, wherethe electric guitars and standard drums may be replaced with horns,keyboards, and bongos or steel drums, and the Punk-Rock rhythm and tempocan be altered to comport with a Reggae rhythm and tempo. Despite suchre-arrangements, the theme and the feeling of the “original” song can bemaintained.

Various embodiments of the present disclosure allow for suchre-arrangement using the augmented reality method and device describedabove. A composer or re-arranger can record individual tracks of a songusing different instruments and/or different styles. Thus, the musicalelements of a particular song or musical composition can be split intotwo independent dimensions: style and instrument. For example one trackmay record a song using a piano. The track may be recorded using thepiano in the same style as the original musical composition or may berecorded using the piano in a different style. The aforementionedphysical image markers can be used to represent these musical elementsas a style-agnostic instrument or as a combination of instrument andstyle.

For example, an original musical composition may be re-arranged using aplurality of musical instruments, where each of the musical instrumentsis associated with a particular music style. That is, a first physicalimage marker 222 can be used to represent an acoustic bass playing thesame musical composition in a Latin style, a second physical imagemarker 224 can be used to represent a piano playing the same musicalcomposition in a Latin style, and a third physical image marker 226 canbe used to represent drums playing the same musical composition in aLatin style (FIG. 2C). Additionally, a fourth physical image marker 228can be used to represent an electric guitar playing a musicalcomposition in a Rock style, and a fifth physical image marker 230 canbe used to represent an electric bass guitar also playing the musicalcomposition in a Rock style (FIG. 2D).

In one embodiment, a physical image marker may be configured with atransmitter (e.g., a radio frequency (RF) transmitter) that sends out asignal encoded with data specifying a type identifier. In such anembodiment, the augmented reality component 202 could receive the signal(e.g., using a receiver or transceiver on the augmented reality device200) and determine the type identifier encoded within the signal. Theaugmented reality component 202 could then determine the type of musicstyle and/or instrument that is represented by that physical imagemarker.

In another embodiment, the physical image marker may contain an embeddedwatermark or other indicia that identifies the type of music styleand/or instrument represented by the physical image marker. Forinstance, the physical image marker could contain a quick response (QR)code that specifies a type identifier corresponding to the type of musicstyle and/or instrument represented by the physical image marker. Moregenerally, however, any type of marker capable of identifying a type ora type identifier may be used. The augmented reality component 202 couldthen detect the embedded marker using one or more cameras 204 of theaugmented reality device 200 and could determine the type of music styleand/or instrument represented by the physical image marker. In aparticular embodiment, the embedded marker is impossible or difficult todetect using the human eye. For example, the embedded marker could beexpressed using an infrared-absorbing material that is invisible ornearly invisible to the human eye.

Upon identifying a physical image marker within the visual scene, theaugmented reality component 202 could then retrieve predefined dataassociated with the identified physical image marker. For example, theaugmented reality component 202 could determine that physical imagemarker 222 is representative of an acoustic bass playing in a Latinstyle. Thus, augmented reality component 202 can retrieve correspondingaugmentation data (i.e., a music track in which an acoustic bass isplaying a musical composition in a Latin style) and present that to theuser via display 206 (and speakers, not shown) of augmented realitydevice 200. That is, the augmented reality component 202 can render aseries of frames depicting an augmented virtual scene based on theaugmentation data. It should be noted that this is performed for allidentified physical image markers simultaneously to allow forsimultaneous play of multiple musical entities. It should be furthernoted that augmented reality component 202 is aware of the currenttemporal position of a music track currently being played such that theintroduction and/or removal of a physical image marker in the visualscene coincides or corresponds to the appropriate portion of the musictrack to be played.

From the user's perspective, the user can choose instruments and stylesindependently and create a newly arranged version of a musicalcomposition. To arrange the song, the user may “insert” one or morephysical image markers into a visual scene. For example, and asillustrated in FIG. 2C, the user may insert physical image markers 222,224, and 226 into the visual scene. The augmented reality component 202of augmented reality device 200 may then identify the physical imagemarker(s) presented in the visual and present the appropriate musicaltracks (previously recorded).

In some embodiments, the user may purchase or obtain the aforementionedmusical tracks from or via the augmented reality application. That is,the user may purchase a set of musical tracks corresponding to a song ormay purchase individual musical tracks according to the desires of theuser. Different styles of musical tracks may also be purchased orotherwise obtained.

It should be noted that the presentation of the appropriate musicaltracks is done in a musically cohesive manner. That is, the individualmusical tracks represented by the physical image markers may play thesame song in the same key and in the same tempo. In this way, anycombination of physical image markers and their corresponding musicaltracks can be used to create an arrangement that is musically pleasing.For example, as illustrated in FIG. 2C, the use of physical imagemarkers 222, 224, and 226 result in an augmented reality version of amusical composition (represented by augmented reality versions 222 v,224 v, and 226 v of physical image markers 222, 224, and 226,respectively) that is presented in a Latin style using a piano track, adrum track, and an acoustic bass track.

Moreover, each physical image marker may be associated with more thanone musical track. For example, within a style, for example, there maybe different versions of that style, whether relating to tempo, someother stylistic variation on the style, or any other musically-relatedvariation. In this way, augmented reality component 202 can choose oneof the plurality of variations at random, in some predetermined ordefault order (where removal and re-entry of the physical image markermay result in the presentation in a different one of the plurality ofvariations), etc. Additionally still, augmented reality component 202may, according to some algorithm or predetermined instructions, play aslightly different musical track (e.g., one of the stylisticsub-variations) if a particular one of another physical image marker ispresent in the visual scene. For example, if a physical image markerrepresentative of a saxophone is present alone (or in the absence of aphysical image marker representative of a lead vocalist), a “flashly”solo Jazz style musical track is played. however, if the physical imagemarker representative of the lead vocalist is identified in the visualscene along with that of the saxophone, a “backup” Jazz style musicaltrack can be played to better complement the lead vocalist musicaltrack.

In some embodiments, game mechanics can be implemented in the augmentedreality application such that the use, movement, placement, etc. of thephysical image markers can be used to teach users about music arranging,for example. That is, the augmented reality application can providevisual and/or audio cues regarding which physical image markers to placein the visual scene, and incorporate one or more teaching elementsassociated therewith. In other embodiments, the user's use, movement,and/or placement of physical image markers can be in the context of agame, where correct usage, movement, and/or placement of particularphysical image markers (considering compatibility of styles or tempi,for example) can be rewarded, or can result in game advancement. Instill other embodiments, when physical image markers are used torepresent musicians, for example, those musicians can be imbued withrelated properties, such as skills regarding rhythm, stamina, precision,etc. A game can involve multiple users or a user and computer player,wherein the user(s) can arrange bands, orchestras, etc. with differentmusicians and/or musical entities with “correct” properties or toprovide the “best sounding” arrangements.

In the example illustrated in FIG. 2D, however, the user has chosen tokeep physical image marker 222 (representative of a piano track in aLatin style) and replace physical image markers 224 and 226 withphysical image markers 228 and 230. Accordingly, the resulting augmentedreality version of the visual scene is output such that the musicalaspect includes a Latin style piano track (via augmented reality version224 v of physical image marker 224), a Rock style electric guitar track(via augmented reality version 228 v of physical image marker 228), anda Rock style electric bass guitar track (via augmented reality version230 v of physical image marker 230). It should be noted that thearrangement or re-arrangement of a musical composition is accomplishedin real or near-real time in accordance with a user's inclusion,removal, and/or replacement of a physical image marker in the visualscene. In this way, there is no latency or delay in the correspondingpresentation of the augmented reality version of the visual scene andaudio to the user.

As should also be appreciated, in addition to identifying a physicalimage marker type, the augmented reality component 202 can usepredefined geometric data associated with the type of musical instrumentto augment the augmented reality version of the instrument's appearance.For instance, and referring to FIG. 2C, such geometric data couldspecify the shape and dimensions of each instrument, and the augmentedreality component 202 could use this information to render framesrealistically depicting a 3D image/rendering of each of the instruments.Moreover, one or more aspects of a physical image marker, such as thedepicted instrument or other aspects (in the example of FIG. 2C, theillustrated notes) can be animated.

As alluded to previously, use of the disclosed embodiments can includeplacement of the physical image markers on a physical augmented realityboard or mat, although the of a reality board or mat is optional. Theaugmented reality board or mat can be used to define an operative areaof the visual scene in which the aforementioned features of musicalcomposition using the physical image markers can be utilized. Like thephysical image markers, one or more aspects of the augmented realityboard or mat can also be presented in an augmented reality fashion.

The position of the physical image marker relative to augmented realitydevice 200 can also be used to control and simulate spatial aspects of amusical composition. That is, and referring back to FIG. 2B, moving aphysical image marker in any one or more directions (x, y, and/or z) canresult in a change in the manner in which the visual and audio aspect(s)of the augmented reality version of a physical image marker ispresented. For example, the closer (along the z-axis) a user brings aphysical image marker to the one or more cameras 204 of augmentedreality device 200, the sound of that instrument may be louder and viceversa. Movement along the x and y axes may also affect the stereo aspectof a composition. Thus, when a user moves a physical image markerfurther to one side or another or up or down relative to the augmentedreality version of the visual scene, the perceived source of the audiowill also move accordingly. This variation in the spatial compositioncan be accomplished while a physical image marker is fully visualized inthe visual scene and/or can be accomplished in conjunction with the usermoving one or more physical image markers in and out of the view of theone or more cameras 204 of augmented reality device 200. Hence, the usercan intuitively explore different spatial compositions to produce aunique arrangement of the song.

It should be noted that the user can record or share arrangementscreated by the user. Moreover, the connectivity aspects of the augmentedreality device 200 can be leveraged to allow for multi-userimplementations of various embodiments of the present disclosure. Forexample, multiple users can collaborate with each other to create uniquemusical arrangements or compositions using their own respectiveaugmented reality devices.

FIG. 3 illustrates an example computing module that may be used toimplement various features of the system and methods disclosed herein,for example, augmented reality device 200 and augmented realitycomponent 202.

As used herein, the term module might describe a given unit offunctionality that can be performed in accordance with one or moreembodiments of the present application. As used herein, a module mightbe implemented utilizing any form of hardware, software, or acombination thereof. For example, one or more processors, controllers,ASICs, PLAs, PALs, CPLDs, FPGAs, logical components, software routinesor other mechanisms might be implemented to make up a module. Inimplementation, the various modules described herein might beimplemented as discrete modules or the functions and features describedcan be shared in part or in total among one or more modules. In otherwords, as would be apparent to one of ordinary skill in the art afterreading this description, the various features and functionalitydescribed herein may be implemented in any given application and can beimplemented in one or more separate or shared modules in variouscombinations and permutations. Even though various features or elementsof functionality may be individually described or claimed as separatemodules, one of ordinary skill in the art will understand that thesefeatures and functionality can be shared among one or more commonsoftware and hardware elements, and such description shall not requireor imply that separate hardware or software components are used toimplement such features or functionality.

Where components or modules of the application are implemented in wholeor in part using software, in one embodiment, these software elementscan be implemented to operate with a computing or processing modulecapable of carrying out the functionality described with respectthereto. One such example computing module is shown in FIG. 3. Variousembodiments are described in terms of this example-computing module 300.After reading this description, it will become apparent to a personskilled in the relevant art how to implement the application using othercomputing modules or architectures.

Referring now to FIG. 3, computing module 300 may represent, forexample, computing or processing capabilities found within aself-adjusting display, desktop, laptop, notebook, and tablet computers;hand-held computing devices (tablets, PDA's, smart phones, cell phones,palmtops, etc.); workstations or other devices with displays; servers;or any other type of special-purpose or general-purpose computingdevices as may be desirable or appropriate for a given application orenvironment. Computing module 300 might also represent computingcapabilities embedded within or otherwise available to a given device.For example, a computing module might be found in other electronicdevices such as, for example navigation systems, portable computingdevices, and other electronic devices that might include some form ofprocessing capability.

Computing module 300 might include, for example, one or more processors,controllers, control modules, or other processing devices, such as aprocessor 304. Processor 304 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 304 is connected to a bus 302, althoughany communication medium can be used to facilitate interaction withother components of computing module 300 or to communicate externally.

Computing module 300 might also include one or more memory modules,simply referred to herein as main memory 308. For example, preferablyrandom access memory (RAM) or other dynamic memory, might be used forstoring information and instructions to be executed by processor 304.Main memory 308 might also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 304. Computing module 300 might likewise include aread only memory (“ROM”) or other static storage device coupled to bus302 for storing static information and instructions for processor 304.

The computing module 300 might also include one or more various forms ofinformation storage mechanism 310, which might include, for example, amedia drive 312 and a storage unit interface 320. The media drive 312might include a drive or other mechanism to support fixed or removablestorage media 314. For example, a hard disk drive, a solid state drive,a magnetic tape drive, an optical disk drive, a compact disc (CD) ordigital video disc (DVD) drive (R or RW), or other removable or fixedmedia drive might be provided. Accordingly, storage media 314 mightinclude, for example, a hard disk, an integrated circuit assembly,magnetic tape, cartridge, optical disk, a CD or DVD, or other fixed orremovable medium that is read by, written to or accessed by media drive312. As these examples illustrate, the storage media 314 can include acomputer usable storage medium having stored therein computer softwareor data.

In alternative embodiments, information storage mechanism 310 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into computing module 300.Such instrumentalities might include, for example, a fixed or removablestorage unit 322 and an interface 320. Examples of such storage units322 and interfaces 320 can include a program cartridge and cartridgeinterface, a removable memory (for example, a flash memory or otherremovable memory module) and memory slot, a PCMCIA slot and card, andother fixed or removable storage units 322 and interfaces 320 that allowsoftware and data to be transferred from the storage unit 322 tocomputing module 300.

Computing module 300 might also include a communications interface 324.Communications interface 324 might be used to allow software and data tobe transferred between computing module 300 and external devices.Examples of communications interface 324 might include a modem orsoftmodem, a network interface (such as an Ethernet, network interfacecard, WiMedia, IEEE 802.XX or other interface), a communications port(such as for example, a USB port, IR port, RS232 port Bluetooth®interface, or other port), or other communications interface. Softwareand data transferred via communications interface 324 might typically becarried on signals, which can be electronic, electromagnetic (whichincludes optical) or other signals capable of being exchanged by a givencommunications interface 324. These signals might be provided tocommunications interface 324 via a channel 328. This channel 328 mightcarry signals and might be implemented using a wired or wirelesscommunication medium. Some examples of a channel might include a phoneline, a cellular link, an RF link, an optical link, a network interface,a local or wide area network, and other wired or wireless communicationschannels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to transitory ornon-transitory media such as, for example, memory 308, storage unit 320,media 314, and channel 328. These and other various forms of computerprogram media or computer usable media may be involved in carrying oneor more sequences of one or more instructions to a processing device forexecution. Such instructions embodied on the medium, are generallyreferred to as “computer program code” or a “computer program product”(which may be grouped in the form of computer programs or othergroupings). When executed, such instructions might enable the computingmodule 300 to perform features or functions of the present applicationas discussed herein.

Although described above in terms of various exemplary embodiments andimplementations, it should be understood that the various features,aspects and functionality described in one or more of the individualembodiments are not limited in their applicability to the particularembodiment with which they are described, but instead can be applied,alone or in various combinations, to one or more of the otherembodiments of the application, whether or not such embodiments aredescribed and whether or not such features are presented as being a partof a described embodiment. Thus, the breadth and scope of the presentapplication should not be limited by any of the above-describedexemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

What is claimed is:
 1. A computer-implemented method, comprising:capturing a visual scene; identifying a physical image marker within thevisual scene, the physical image marker representing a first musicalentity; augmenting the visual scene with the augmented reality versionof the physical image marker; outputting an augmented reality version ofthe visual scene for display as part of a computerized game inconjunction with audio output based upon a first portion of a musicalarrangement of a musical composition, the first portion of the musicalarrangement comprising a first musical style and sound characteristicsassociated with the first musical entity, the musical arrangement beingre-arrangeable within the augmented reality version of the visual scene;and advancing gameplay within the computerized game upon a determinationthat the musical arrangement maintains a musical theme and impression ofthe musical composition.
 2. The computer-implemented method of claim 1further comprising, presenting at least one of audio and visual cuesregarding at least one of movement and placement of the physical imagemarker within the visual scene.
 3. The computer-implemented method ofclaim 2, wherein the at least one of audio and visual cues reflects theat least one of the movement and placement of the physical image markercomporting with the maintenance of the musical theme and impression ofthe musical composition.
 4. The computer-implemented method of claim 1further comprising, determining the maintenance of the musical theme andimpression of the musical composition by determining compatibility of atleast one of the first musical style and tempo of the first musicalstyle with the musical theme and impression of the musical composition.5. A computer-implemented method, comprising: capturing a visual scene;identifying a first set of physical image markers within the visualscene, the first set of physical image markers representing a first setof musical entities; augmenting the visual scene with augmented realityversions of the first set of physical image markers; outputting anaugmented reality version of the visual scene for display as part of acomputerized game in conjunction with audio output based upon a firstmusical arrangement of a musical composition, the first musicalarrangement comprising a first musical style and sound characteristicsassociated with the first set of musical entities, the first musicalarrangement being re-arrangeable within the augmented reality version ofthe visual scene; comparing the first musical arrangement with a secondmusical arrangement; and determining a preferred musical arrangementbetween the first musical arrangement and the second musicalarrangement.
 6. The computer-implemented method of claim 5 furthercomprising, determining the first musical arrangement based upon atleast one of movement and placement of the first set of the physicalimage markers representing the first set of musical entities.
 7. Thecomputer-implemented method of claim 5, further comprising: augmentingthe visual scene with an augmented reality version of a second set ofphysical image markers representing a second set of musical entities;and outputting the augmented reality version of the visual scene fordisplay as part of the computerized game in conjunction with additionalaudio output based upon the second musical arrangement of the musicalcomposition.
 8. The computer-implemented method of claim 7, wherein thesecond musical arrangement comprises a second musical style and soundcharacteristics associated with a second set of musical entities, thesecond musical arrangement being re-arrangeable within the augmentedreality version of the visual scene.
 9. The computer-implemented methodof claim 7 further comprising, determining the second musicalarrangement based upon at least one of movement and placement of thesecond set of the physical image markers representing the second set ofmusical entities.
 10. The computer-implemented method of claim 5 furthercomprising, determining the preferred musical arrangement based upon adetermination that the first or second musical arrangement maintains amusical theme and impression of the musical composition.
 11. Anapparatus, comprising: at least one camera capturing a visual scene; anaugmented reality component identifying a physical image marker withinthe visual scene, the physical image marker representing a first musicalentity, and augmenting the visual scene with the augmented realityversion of the physical image marker; a display displaying an augmentedreality version of the visual scene; and one or more speakers outputtingaudio in conjunction with the augmented reality version of the visualscene and based upon a first portion of a musical arrangement of amusical composition, the first portion of the musical arrangementcomprising a first musical style and sound characteristics associatedwith the first musical entity, the musical arrangement beingre-arrangeable within the augmented reality version of the visual scenewhile maintaining a musical theme and impression of the musicalcomposition.
 12. The apparatus of claim 15, wherein the augmentedreality component identifies the physical image marker by one ofreceiving an identification signal from a radio frequency transmitterimplemented within the physical image marker or by sensing an indicatorembedded in the physical image marker.
 13. The apparatus of claim 15,wherein the augmented reality component retrieves a pre-recorded musicaltrack representative of the first portion of the musical arrangementidentified by identifying the physical image marker, the pre-recordedmusical track comprising the audio output by the one or more speakers.14. The apparatus of claim 15, wherein the augmented reality componentrenders at least one of a 3D representation of one or more aspects ofthe physical image marker and an animated representation of the one ormore aspects of the physical image marker.
 15. The apparatus of claim15, wherein the one or more speakers output additional audio inconjunction with the augmented reality version of the visual scene andbased upon a second portion of the musical arrangement and soundcharacteristics associated with a second musical entity.
 16. Theapparatus of claim 15, wherein the re-arrangement of the musicalcomposition comprises a plurality of different musical styles and aplurality of musical entities, and wherein the different musical stylesand the plurality of musical entities utilize the same tempo and key.